Light Textual Inference for Semantic Parsing

There has been a lot of recent interest in Semantic Parsing, centering on using data-driven techniques for mapping natural language to full semantic representations (Mooney, 2007). One particular focus has been on learning with ambiguous supervision (Chen and Mooney, 2008; Kim and Mooney, 2012), where the goal is to model language learning within broader perceptual contexts (Mooney, 2008). We look at learning light inference patterns for Semantic Parsing within this paradigm, focusing on detecting speaker commitments about events under discussion (Nairn et al., 2006; Karttunen, 2012). We adapt PCFG induction techniques (Börschinger et al., 2011; Johnson et al., 2012) for learning inference using event polarity and context as supervision, and demonstrate the effectiveness of our approach on a modified portion of the Grounded World corpus (Bordes et al., 2010). KEYWORDS: Semantic Parsing, Computational Semantics, Detecting Textual Entailment, Grammar Induction. 1 Overview and Motivation Semantic Parsing is a subfield in NLP that looks at using data-driven techniques for mapping language expressions to complete semantic representations (Mooney, 2007). A variety of corpora and learning techniques have been developed for these purposes, both for doing supervised learning (Kate et al., 2005; Kwiatkowski et al., 2010) and learning in more complex (ambiguous) settings (Chen and Mooney, 2008, 2011). In many studies, the learning is done by finding alignments between (latent) syntactic patterns in language and parts of the target semantic representations, often using techniques from Statistical Machine Translation (Wong and Mooney, 2006; Jones et al., 2012). Despite achieving impressive results in different domains, learning semantic inference patterns is often not addressed, making it unclear how to apply these methods to tasks like Detecting Textual Entailment. In this work, we show how to learn light (syntactic) inference patterns for textual entailment using loosely-supervised Semantic Parsing methods. Detecting Textual Entailment is a topic that has received considerable attention in NLP, largely because of its connection to applications such as question answering, summarization, paraphrase generation, and many others. The goal, loosely speaking, is to detect entailment inference relationships between pairs of sentences (Dagan et al., 2005). More recent work on Hedge and Event Detection (Farkas et al., 2010) has focused on similar issues related to determining event certainty, especially in the biomedical domain (Example 3 (Thompson et al., 2011)). Four inferences are shown in Examples 1-4, and relate to implied speaker commitments (Karttunen, 2012; Nairn et al., 2006) about events under discussion. 1. John forgot to help Mary organize the meeting (a) |= John didn’t help Mary organize the meeting 2. John remembered (to not neglect) to turn off the lights before leaving work (a) |= John turned off some lights 3. NF-kappa B p50 alone fails to (=doesn’t) stimulate kappa B-directed transcription 4. The camera {didn’t manage, managed} to impress me (=negative/positive opinion) In Example 1, the speaker of the sentence is committed to the belief that the main event (i.e. helping Mary organize the meeting) did not occur, whereas the opposite is true in Example 2. This is triggered by the implicative phrases Forget to X and Remember to X, which affect the polarity of the modified event X. These inferences relate to the semantics of English complement constructions, a topic well studied in Linguistics (Karttunen, 1971; Kiparsky and Kiparsky, 1970). They are also part of a wider range of inference patterns that are syntactic in nature, or visible from language surface form (Dowty, 1994). They have been of interest to studies in proof-theoretic semantics and Natural Logic, which look at doing inference on natural language directly (MacCartney and Manning, 2007; Moss, 2010; Valencia, 1991). We aim to learn these implicative patterns, building on existing computational work. (Nairn et al., 2006; Karttunen, 2012) provide a classification of implicative verbs according to the effect they have on their surrounding context. They observe that implicative constructions differ in terms of the polarity contexts they occur in, and the effect they have in these contexts. As illustrated in Table 1, one-way implicatives occur in a single polarity, whereas two-way implicatives occur in both. For example, Forget to X in Example 1 switches polarity in a positive context to negative, and has the opposite effect in a negative context, giving it the implicative signature (+)(-), (-)(+) (i.e. start context, result). Implicatives can be productively stacked together as shown in Example 2. Determining the resulting inference for an arbitrary nesting of implicatives requires computing the relative polarity of each smaller phrase, which is the idea behind the polarity propagation algorithm (Nairn et al., 2006). This can be done directly from syntax by traversing a tree annotated with polarity information and calculating the polarity interactions incrementally. This general strategy for doing inference, which relies on syntactic and lexical features alone, avoids a full semantic analysis and translation into logic (Bos and Markert, 2005), and has been successfully applied to more general textual entailment tasks (MacCartney and Manning, 2007, 2008). One problem with the approach of (Nairn et al., 2006), however, is that the implicative signatures of verbs must be manually compiled, as there are no standard datasets available for doing learning. To our knowledge, there has been little work on learning these specific patterns (some related studies (Danescu-Niculescu-Mizil et al., 2009; Cheung and Penn, 2012)), which would be useful for applying these methods to languages and domains where resources are not available. Further, their algorithm encodes the lexical properties as hard facts, making it hard to model potential uncertainty and ambiguity associated with these inferences (e.g. if John was able to do X, how certain are we that he actually did X?) The semantics of implicative expressions can often be inferred from non-linguistic context. Knowing that managed to X implies X is something we can learn from hearing this utterance in contexts where X holds. Recent studies on learning from ambiguous supervision for Semantic Parsing (Chen and Mooney, 2008, 2011) has looked at incorporating perceptual context (Mooney, 2008) of this sort into the learning process (see also (Johnson et al., 2012)). Work on the Sportscaster Corpus (Chen and Mooney, 2008) considers interpreting soccer commentary in ambiguous contexts where several closely occurring events are taking place. Their data is taken from a set of simulated soccer games extended with human commentary. Each comment is paired with a set of grounded events occurring in the game around the time of the comment. Using these ambiguous contexts as supervision, they learn how to map novel sentences to the correct grounded semantic representations. We look at learning implicative inference in a similar grounded learning scenario, using ambiguous contexts and the polarity of events as supervision. We use a modified portion of the Grounded World corpus (Bordes et al., 2010), which was extended to have phrasal implicatives and ambiguous contexts. Three training examples are displayed in Figure 1, and an illustration of the analysis we aim to learn. Each example is situated in a virtual house environment and a context, and describes events taking place in the house. Details of the corpus and learning procedure are described in the next section. 2 Experiments 2.1 Materials The original Grounded World corpus (Bordes et al., 2010) is a set of English descriptions situated within a virtual house, and was designed for doing named entity recognition and situated pronoun resolution. Inside the house is a fixed set of domain objects, including a set of actors (e.g. father, brother), a set of furniture pieces (e.g. couch, table), a set of rooms (e.g. Type Examples Effect on Polarity Two-way implicatives manage to (+)(+) | (–)(–) forget to (+)(–) | (–)(+) One-way +implicatives force to (+)(+) refuse to (+)(–) One-way -implicatives attempt to (–) (–) hesistate to (–)(+) Table 1: Types of Implicative Verbs from (Nairn et al., 2006; Karttunen, 2012) # Sentences # Token Gold Relations Aver. Context Size 7,010 Total (6,065 (85%) unique) 2,444 (63 unique concepts) 2.17 (90% > 1) 1,863 Implicative Sentences (26%) Frequent Verb Tokens: refuse to, manage to, decline to, admit to, remember to, dare to Complex Constructions: fail to neglect to, didn’t refrain from, refuse to remember to Examples: Their grandmother [admitted++ to]+ drinking a little wine. The brother [didn’t+− dare++ to]− move into the bedroom. Their mom [remembered++ to not−+ forget+− to]+ grab their toy from the closet Table 2: Details of the extended Grounded World Corpus. The average context size is the average number of events in the ambiguous training contexts. On the bottom are some corpus examples with implicative constructions. living room, bathroom), and a set of small objects (e.g. doll, chocolate), plus a set of 15 event types (e.g. eating, sleeping, and drinking). For our study, we used a subset of 7,010 examples from the original training set, and modified the sentences to have syntactic alternations and paraphrases not seen in the initial corpus. 1,863 of these sentences were modified to have implicative constructions (using 70 unique constructions from 20 verb types, see examples in Table 2)1 that relate to the original content of the sentence, in some cases creating negated forms of the original sentences. We expanded the original named-entity annotations to normalized semantic representations, and produced a set of distractor events (or observable contexts) for each example to make the data ambiguous. Three training examples are shown in Figure 1. In the first example, the sentence is situated in three observable events (sleeping, getting and bringing). These can be viewed as events in the current context or the speaker’s belief state. Additional information about the world state (i.e. location of objects) is provided from the original corpus for pronoun resolution, which we ignore. The last two examples have implicative constructions, the first one leading to a negative inference (the sister is not sleeping in the bedroom/guestroom). The last example leads to a positive inference (the sister got a toy from the closet/storage). We show the annotations from the original corpus for comparison. Expanding the relations from the overall corpus and situating them within ambiguous contexts 1we used the phrasal implicative lexicon available at http://www.stanford.edu/group/csli_lnr/Lexical_Resources /phrasal-implicatives/, compiled by the authors of (Nairn et al., 2006; Karttunen, 2012) U"erance: 18146 while he is sleeping in the bedroom Original Annota0on*: -­‐ -­‐ -­‐ -­‐ Observable Context 18146: (bring friend, water, (toLoc bedroom)) (get baby, videogame) (sleep friend, (loc bedroom)) World State: (in-­‐bedroom ‘(bed, closet, friend, ...)) (in-­‐kitchen ‘(baby,closet, friend, ...)) ..... U"erance: 50034 the sister failed to nap in the guestroom Original Annota0on*: -­‐ -­‐ -­‐ -­‐ -­‐ Observable Context 28932: (move hamster, (toLoc office)) (play hamster) (neg (sleep sister, (loc bedroom))) U"erance: 7054 the sister didn’t fail to get their toy from the storage Observable Context 7054: (get baby, doll), (get sister, gtoy, (fromLoc closet)) Latent seman0c analysis GET’(+) FromLOC(CLOSET)’ from the storage GTOY’ their toy SISTER’ the sister (get sister, gtoy, (fromLoc closet))